Papers with mean reciprocal rank
Voice Query Auto Completion (2021.emnlp-main)
Copied to clipboard
Raphael Tang, Karun Kumar, Kendra Chalkley, Ji Xin, Liming Zhang, Wenyan Li, Gefei Yang, Yajie Mao, Junho Shin, Geoffrey Craig Murray, Jimmy Lin
| Challenge: | Existing methods fail to complete voice queries from incomplete prefixes because they use orthographic prefix and substrings instead of the true phonetic prefix. |
| Approach: | They propose to condition QAC approaches on intermediate transcriptions to complete voice queries. |
| Outcome: | The proposed method obtains an 18% relative improvement over previous methods on a speech-enabled smart television with real-life voice search traffic. |
DiAL : Diversity Aware Listwise Ranking for Query Auto-Complete (2024.emnlp-industry)
Copied to clipboard
| Challenge: | Query Auto-Complete (QAC) is an essential search feature that helps users articulate their query by suggesting relevant completions as they type. |
| Approach: | They propose a new framework that explicitly optimizes for diversity alongside customer feedback signals to balance relevance and diversity. |
| Outcome: | The proposed framework yields an improvement of 8.5% in MRR and 22.8% in NDCG compared to the pairwise ranking approach on an eCommerce dataset. |
BERT Is Not The Count: Learning to Match Mathematical Statements with Proofs (2023.eacl-main)
Copied to clipboard
| Challenge: | Existing work on mathematical article analysis uses natural language processing to solve complex mathematical articles. |
| Approach: | They propose a bilinear similarity model and two decoding methods to match statements to proofs effectively. |
| Outcome: | The proposed model matches proofs to statements without being aware of proofs, but it follows a relatively shallow symbolic analysis and matching to achieve that performance. |
Ensemble of MRR and NDCG models for Visual Dialog (2021.naacl-main)
Copied to clipboard
| Challenge: | BLEU scores favor correct syntax over semantics. |
| Approach: | They propose a non-parametric ranking method that integrates the ranks of two strong MRR and NDCG models into a single ranking that excels on both metrics. |
| Outcome: | The proposed model can keep the MRR and NDCG models state-of-the-art and the NDGC models state of the art. |
SimKGC: Simple Contrastive Knowledge Graph Completion with Pre-trained Language Models (2022.acl-long)
Copied to clipboard
| Challenge: | Text-based methods lag behind graph embedding-based approaches for knowledge graph completion (KGC) |
| Approach: | They propose three types of negatives to improve contrastive learning to improve learning efficiency. |
| Outcome: | The proposed model outperforms embedding-based methods on several benchmark datasets. |
TEMP: Taxonomy Expansion with Dynamic Margin Loss through Taxonomy-Paths (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing taxonomies are unable to maintain coverage due to the rising of new concepts . TEMP uses pre-trained contextual encoders to predict the position of new ideas . |
| Approach: | They propose a self-supervised taxonomy expansion method that ranks taxonomies by ranking them . they use pre-trained contextual encoders to train the model with dynamic margin loss . |
| Outcome: | The proposed method outperforms state-of-the-art taxonomy expansion methods by 14.3% and 15.8% on public benchmarks. |
Knowledge Base Completion Meets Transfer Learning (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing knowledge bases are tedious and require a large amount of labor to build. |
| Approach: | They propose a method that allows for transfer of knowledge from one collection of facts to another without entity or relation matching. |
| Outcome: | The proposed method is the most impactful on small datasets, showing a 6% increase in rank and 65% decrease in rank over the previous best method. |
Low-Dimensional Hyperbolic Knowledge Graph Embeddings (2020.acl-main)
Copied to clipboard
| Challenge: | Existing methods for predicting missing facts do not account for hierarchical and logical patterns in KGs. |
| Approach: | They propose a class of hyperbolic KG embedding models that capture hierarchical and logical patterns. |
| Outcome: | Experimental results show that the proposed method improves by 6.1% in mean reciprocal rank in low dimensions over previous methods. |